Fundamental Distortion Limits of Analog-to-Digital Compression

A theory of minimizing distortion in reconstructing a stationary signal under a constraint on the number of bits per sample is developed. We first analyze the optimal sampling frequency required in order to achieve the optimal distortion-rate tradeoff for a stationary bandlimited signal. To this end, we consider a combined sampling and source coding problem in which an analog Gaussian source is reconstructed from its encoded sub-Nyquist samples. We show that for processes whose energy is not uniformly distributed over the spectral band, each point on the distortion-rate curve of the process corresponds to a sampling frequency smaller than the Nyquist rate. This characterization can be seen as an extension of the classical sampling theorem for bandlimited random processes in the sense that it describes the minimal amount of excess distortion in the reconstruction due to lossy compression of the samples, and provides the minimal sampling frequency fDR required in order to achieve that distortion. We compare the fundamental limits of combined source coding and sampling to the performance in pulse code modulation (PCM), where each sample is quantized by a scalar quantizer using a fixed number of bits.